Skip to content

Keep data source attributes in the source lookup and in keep_latest_version - #2867

Open
sankalpsthakur wants to merge 3 commits into
FlexMeasures:mainfrom
sankalpsthakur:fix/2505-data-source-attributes
Open

sankalpsthakur wants to merge 3 commits into
FlexMeasures:mainfrom
sankalpsthakur:fix/2505-data-source-attributes

Conversation

@sankalpsthakur

Copy link
Copy Markdown
Contributor

Summary

  • get_data_source() only looks for the source without attributes, which is the kind it creates, so an attribute-carrying sibling no longer makes it raise MultipleResultsFound.
  • keep_latest_version() groups sources by (name, type, model, attributes), so sources that differ only by their attributes each keep their data, with versioning still applied within each group. The aggregator's missing-members attribute does not count, so a complete aggregate still replaces its partial one.
  • This answers the question in the issue as: attributes are part of a source's identity for read-time precedence, except the missing members of a partial aggregate. An existing test comment said sources differing by data_generator_config collapse to the newest one; the reference test is updated to the new grouping.
  • _select_latest_version_and_belief_per_event() in time_series.py still groups without attributes. I left it for a follow-up, happy to add it here if you prefer.

Validation

  • New tests for both cases, and the reference implementation in the equivalence test now groups the same way. With the fix reverted, these 3 tests fail.
  • pytest flexmeasures/data/tests/test_data_source.py flexmeasures/data/tests/test_search_postprocessing.py: 49 passed. flexmeasures/data/models/reporting/tests and flexmeasures/data/tests/test_queries.py: 101 passed.
  • black --check and flake8 clean on the changed files.

Fixes #2505

AI/LLM disclosure

AI tools assisted with this change; I reviewed it and ran the tests above.

…nd when keeping the latest version

get_data_source() now looks for the source without attributes, which is the one it creates,
so a sibling recording attributes under the same name, model, version and type no longer makes it raise MultipleResultsFound.

keep_latest_version() now groups sources by their attributes as well as their name, type and model,
so sources differing only by attributes each keep their data. The members a partial aggregate names as missing
do not count, so a complete aggregate still replaces its partial one.

Fixes FlexMeasures#2505

Signed-off-by: Sankalp Thakur <[email protected]>
@read-the-docs-community

read-the-docs-community Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 flexmeasures | 🛠️ Build #35916298 | 📁 Comparing 501b4fe against latest (8d59a64)

  🔍 Preview build  

5 files changed · ± 5 modified

± Modified

Signed-off-by: Sankalp Thakur <[email protected]>

# Conflicts:
#	documentation/changelog.rst

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Source identity ignores attributes in two places: get_data_source() raises, keep_latest_version() hides data

1 participant